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Thresholds  may  be  chosen  for  images  containing  several  object 
classes  by  clustering  thinned  edge  points  in  a 2-D  histogram, 
whose  axes  represent  gray  level  value  and  edge  value.  Each 
such  edge  cluster  suggests  its  average  gray  level  as  a 
threshold.  Interior  clusters  may  also  be  defined  as  represen- 
tatives of  object  class  interiors.  The  relation  of  edge 
clusters  to  interior  clusters  gives  rise  to  a classification 
strategy  based  on  partitioning  the  2-D  histogram  into  disjoint 
regions  labelled  as  to  object  class.  Each  partition  is  a 
classification  domain  for  points  of  the  original  gray  level 
image . 
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Introduction 
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A variety  of  methods  for  selecting  thresholds  are  known 
[1] . Many  of  them  attempt  to  deduce  the  best  threshold  by 
studying  properties  associated  with  points  on  the  boundary  be- 
tween object  and  background.  Thus  if  the  object  is  substan- 
tial and  contrasts  with  the  background,  then  the  image's  gray 
level  histogram  will  exhibit  a valley  at  the  gray  level 
associated  with  border  points.  More  complicated  schemes  study 
the  cooccurrence  of  high  edge  value  and  gray  level  as  repre- 
sented by  a two-dimensional  histogram.  It  has  been  shown  that 
for  images  containing  one  object  class  and  one  background 
class,  the  average  gray  level  of  high  edge  value  points  pre- 
dicts a good  threshold  [2].  This  scheme,  however,  fails  for 
images  having  several  object  classes,  since  high  edge  values 
may  arise  from  the  adjacencies  among  several  gray  level  popula- 
tions each  requiring  a different  threshold.  In  what  follows, 
we  present  an  approach  to  thresholding  in  a multi-population 
environment. 
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2 . Edge  detection  and  thinning 

Our  approach  is  to  produce  clusters  of  points  correspond- 
ing to  region  borders  and  to  associate  the  average  gray  level 
of  each  cluster  with  a threshold  for  the  corresponding  region. 
Region  borders  usually  correspond  to  points  of  locally 
maximum  edge  response.  Our  previous  work  suggests  the  use  of 
edge  detectors  which  select  at  each  point  the  maximum  differ- 
ence of  averages  of  adjacent  neighborhoods  over  several  direc- 
tions [3] . By  suppressing  non-maximum  responses  normal  to  the 
selected  direction  (i.e.,  across  the  edge),  thin  contours 
result  which  appear  to  surround  object  regions.  Figure  lb 
shows  unthinned  edge  detector  response;  Figure  Ic  illustrates 
the  results  of  thinning  using  non-maximum  suppression. 

This  process  produces  as  a by-product  points  with  very 
low  edge  value,  including  values  which  truncate  to  zero.  Such 
points  correspond  to  the  interiors  of  homogeneous  regions. 

It  is  useful  to  include  such  points  in  our  analysis,  even 
though  they  constitute  a population  with  fundamentally  differ- 
ent properties  from  contour  points.  Figure  2 illustrates 
thinned  detector  responses  with  region  interior  maxima  in- 
cluded. 


3.  Construction  of  a two-dimensional  histogram 

Once  the  thinning  has  been  accomplished,  the  resulting 
parts  can  be  accumulated  in  a two-dimensional  histogram.  The 
2-D  histogram  consists  of  two  labeled  coordinate  axes.  The 
horizontal  axis  represents  the  gray  level  value  of  the  point; 
the  vertical  axis  represents  the  edge  value.  Each  point 
which  survives  the  thinning  procedure  is  accumulated  in  the 
histogram  at  its  (gray  level,  edge  value)  coordinates.  Figure 
3 shows  examples  of  images  together  with  their  2-D  histograms. 

Consider  the  noisy  synthetic  image  in  Figure  3a  and  its 
2-D  histogram  in  Figure  3b.  The  three  clusters  lying  along 
the  top  of  the  histogram  have  very  low  edge  values.  These 
clusters  are  centered  at  gray  levels  20,  30,  and  40  from  left 
to  right  and  represent  interiors  of  the  three  distinct  re- 
gions of  the  picture.  The  cluster  lying  between  gray  levels 
30  and  40  and  centered  at  edge  value  8 represents  the  boundary 
between  the  inner  disk  and  the  surrounding  annulus.  Similarly 
the  cluster  lying  between  gray  levels  20  and  40  and  centered 
at  edge  value  15  represents  the  boundary  between  the  annulus 
and  the  background.  The  three  clusters  at  the  top  of  the  2-D 
histogram  are  called  interior  clusters  since  they  represent 
the  interiors  of  regions.  The  other  two  clusters  are  called 
edge  clusters  since  they  represent  boundaries  between  regions. 

In  practice,  it  was  found  that  the  gray  level  of  a 
thinned  edge  point  does  not  serve  as  a good  coordinate  for 
the  2-D  histogram.  Specifically,  for  a step  edge,  the  gray 
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level  of  an  edge  point  lies  in  one  population  or  the  other  but 
not  at  any  intermediate  gray  value.  The  profile  of  a sharp 
edge  is  shown  in  Figure  4a.  If  an  edge  operator  is  applied 
such  that  the  difference  of  neighborhoods  A and  B is  calculated, 
the  gray  level  of  the  points  will  have  either  the  value  of 
the  points  in  A or  the  value  of  the  points  in  B.  Therefore 
the  2-D  histogram  will  be  as  shown  in  Figure  4b.  The  sharp 
edge  is  not  displayed  as  a cluster,  but  rather  as  two  spikes 
where  one  spike  is  plotted  at  gray  level  a for  points  in  A 
and  the  other  at  gray  level  b for  points  in  B.  Figure  5 
shows  a test  pattern  and  the  2-D  histogreun  resulting  from 
accumulating  points  based  on  gray  level  and  edge  value.  Note 
that  the  histogram  consists  of  a pair  of  spikes  corresponding 
to  the  gray  levels  on  either  side  of  the  step  edge. 

In  order  to  merge  the  two  spikes  into  a single  cluster 
for  sharp  edges,  the  gray  level  coordinate  of  the  2-D  histo- 
gram is  replaced  by  the  average  gray  level  across  the  direc- 
tion of  the  edge.  Since  edge  value  is  obtained  as  usual  by 
differencing  over  neighborhoods,  the  average  gray  level  is 
obtained  by  computing  average  gray  level  over  these  same 
neighborhoods.  In  this  way  points  near  neighborhoods  A and 
B of  Figure  4a  will  be  mapped  near  as  shown  in  Figure 

4c.  Figures  5c  and  6c  show  the  effect  on  the  2-D  histo- 
gram of  using  the  average  gray  level  instead  of  the  point 
gray  level. 


In  addition  to  creating  clusters  from  values  at  sharp 
edges,  the  2-D  histogram  using  averaging  will  also  tend  to 


make  clusters  more  compact  in  the  direction  of  the  horizontal 
axis.  The  reason  can  be  illustrated  by  the  profile  of  an 
edge  shown  in  Figure  7.  In  a 2-D  histogram  not  using  averaging, 
the  point  x will  get  mapped  into  gray  level  z.  However,  since 
the  neighborhood  A contains  points  where  the  edge  ramp  begins 
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to  level  off,  the  value  — ^ will  be  greater  than  z and  therefore 
points  near  x will  be  mapped  at  higher  gray  level  values  in  the  2-D 
histogram  using  averaging.  The  reverse  happens  for  points 
near  the  top  of  the  ramp,  i.e.,  they  will  get  mapped  to  lower 
gray  level  values.  Therefore  the  cluster  tends  to  contract 
in  the  horizontal  direction  since  points  are  mapped  closer 
to  the  center  of  the  cluster. 

It  is  important  to  note  that  the  size  of  a cluster  (i.e., 
the  number  of  points  in  it)  is  closely  related  to  properties 
of  the  region  it  describes.  Thus  interior  clusters  relate 
both  to  the  area  of  the  region  and  to  the  size  of  the  neighbor- 
hood over  which  the  local  operations  (edge  detection,  non- 
maximum suppression)  are  defined.  For  small  object  regions, 
there  may  be  points  sufficiently  far  from  the  object  bound- 
ary to  resist  suppression.  Thus  interior  clusters  may  be  in- 
distinguishable from  noise,  or  may  be  nonexistent. 

Clusters  of  points  at  higher  edge  values  are  more  likely 
to  be  significant  (based  on  our  homogeneity  assumptions) . 

The  size  of  an  edge  cluster  is  therefore  related  to  the 
perimeter  of  the  surrounded  region  in  the  image.  Since  peri- 
meter increases  (roughly,  for  digital  images)  as  the  square 
root  of  area,  the  edge  clusters  for  objects  of  moderately 
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4 . Relationships  between  clusters  in  a 2-D  histogram 

Consider  the  edge  cluster  in  Figure  8 whose  centroid  is 
(e,g).  The  value  g is  approximately  equal  to  the  average 
gray  value  of  the  edge  ramp  along  the  contour  between  the  two 
interior  regions.  The  value  e is  approximately  equal  to  the 
average  thinned  edge  response  along  this  contour.  It  is 
limited  by  the  height  of  the  edge  ramp,  i.e.,  in  general,  it 
will  not  exceed  the  value  of  this  height.  Furthermore,  the 
value  of  e depends  on  the  relationship  between  the  width  of 
the  edge  ramp  and  the  neighborhood  size  used  by  the  edge 
operator.  If  the  neighborhood  size  is  less  than  the  edge 
ramp  width,  the  value  of  e will  be  less  than  the  edge  ramp 
height  (see  Figure  9a) . 

The  edge  cluster  in  Figure  8 serves  as  the  boundary 
separating  two  regions  of  average  gray  level  g-e/2  and  g+e/2 
in  the  original  image.  This  relation  of  an  edge  cluster  to 
two  interior  clusters  has  several  consequences.  Finding  two 
interior  clusters  at  gray  levels  g-e/2,  g+e/2,  respectively, 
would  serve  as  confirmation  of  this  assertion.  Conversely, 
to  determine  whether  two  regions  with  average  gray  level  g^, 
g2,  respectively,  share  a common  boundary  (i.e.,  are  adjacent), 
one  could  attempt  to  locate  an  edge  cluster  with  centroid 
(|g2-gil  , (g]^+g2)/2)»  as  in  Figure  10.  Finally,  finding  an 
edge  cluster  with  centroid  (e,g)  and  one  interior  cluster  at 
gray  level  g-e/2  (see  Figure  8 ) would  serve  to  suggest  an- 


tence.  It  may  be  indiscernible  either  because  the  interior 
region  has  no  points  with  near-zero  gradient  or  the  interior 
cluster  cannot  be  separated  from  another  cluster. 


^ In  general,  however,  e < Ig2-g2^1,  i.e.,  the  average  edge 

I value  of  edge  clusters  is  somewhat  lower  than  predicted, 

i Figure  9b  shows  the  edge  ramp  and  neighborhoods  at  P over 

i 

!>  which  the  edge  values  are  computed.  Note  that  the  average 

I value  a (over  A)  is  generally  less  than  the  value  of  g2» 

\ while  the  average  value  b (over  B)  is  generally  greater  than 

► 

the  value  of  g^^.  Thus,  in  general,  | a-b | < | g2~g^  1 . We  can 
now  more  accurately  predict  the  location  of  cluster  centroids 

1 

based  on  a knowledge  of  the  edge  ramp  and  the  edge  detector. 

Thus,  for  an  edge  detector  using  differences  of  averages  over 

sxs  neighborhoods  and  for  a linear  edge  ramp  of  height  h and 

width  w,  the  expected  maximal  edge  response  is  h - 

Whereas  h is  normally  unconstrained,  w,  the  ramp  width  (or 

edge  fuzziness) , is  fairly  constant  within  an  image  and  there- 

w 

fore  the  edge  operator  size  s should  be  chosen  so  that  jg- 
is  small. 
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5. 


Cluster  extraction 


We  have  investigc  ed  some  simple  methods  of  cluster  ex- 
traction, and  we  now  describe  one  which  has  been  moderately 
successful.  First,  use  the  histogram  of  thinned  edge  values 
(which  is  nothing  more  than  the  projection  of  the  2-D  histo- 
gram on  the  edge  axis)  to  detect  edge  value  ranges  containing 
significant  peaks.  Many  1-D  histogram  segmentation  schemes 
exist  [1].  A conservative  method  is  best.  Each  of  these 
ranges  corresponds  to  a horizontal  strip  across  the  2-D  histo- 
gram (see  Figure  11)  . 

For  each  strip,  construct  a standard  gray  value  histogram, 
i.e.,  project  each  individual  strip  onto  the  x-axis.  Segment 
each  such  histogram  as  before  according  to  its  peaks.  Each 
such  segment  corresponds  to  a rectangle  in  the  original  2-D 
histogram  (Figure  11)  . Clusters  are  associated  with  well- 
populated  rectangles.  Thresholds  may  then  be  computed  as  aver- 
age gray  levels  within  clusters. 
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Coordinating  multiple  thresholds 

Given  a set  of  thresholds  for  an  image  obtained  by 
locating  the  centroid  gray  levels  for  the  edge  clusters,  it 
is  unclear  how  one  applies  them  in  general.  For  example,  con- 
sider the  drawing  in  Figure  12a  and  its  2-D  histogram.  Figure 
12b.  The  center  of  the  edge  cluster  belonging  to  the  in- 
terior clusters  at  gray  levels  30  and  40  is  at  gray  level  35; 
while  the  edge  cluster  separating  the  interior  clusters  at  20 
and  40  has  30  as  its  center.  Thus  the  thresholds  are  30  and 
35.  The  threshold  at  30  will  optimally  separate  the  back- 
ground from  the  outer  boundary  of  the  ring;  however,  it  will 
cause  the  hole  in  the  ring  to  break  up  in  a random  fashion. 

The  threshold  at  35  will  in  fact  separate  the  hole  from  the 
ring  but  will  assign  too  many  border  points  of  the  20-40 
border  to  the  background  region.  Thus  neither  threshold  is 
by  itself  optimal. 

A solution  to  this  problem  can  be  obtained  by  partitioning 
the  2-D  histogram  into  disjoint  regions  which  are  labeled 
as  to  object  class  (Figure  13) . Thus  all  points  in  the 
original  image  as  (opposed  to  thinned  points  only) would  be 
classified  based  on  the  feature  pair  (gray  level,  edge  value). 
The  location  of  each  feature  pair  in  the  partitioned  histo- 
gram would  determine  the  object  class  to  which  each  image 
point  belongs.  Note  that  each  half  of  an  edge  cluster  belongs 
to  a different  interior  cluster  (see  the  arrows  in  Figure  13) . 

A simple  algorithm  to  perform  this  partitioning  on  a complete 
well-formed  2-D  histogram  is  as  follows: 
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Draw  vertical  lines  midway  between  each  adjacent  pair 
of  interior  clusters.  The  lines  extend  to  the  bottom 
of  the  histogram.  Starting  at  the  lowest  edge  value 
edge  cluster,  draw  a vertical  line  bisecting  the 
cluster  and  extending  from  just  above  the  cluster  to 
the  bottom  of  the  histogram.  Draw  the  (horizontal) 
perpendicular  to  the  cluster  bisector  at  the  endpoint 
just  above  the  cluster  and  extend  it  until  it  meets 
the  vertical  on  each  side.  Delete  each  of  those  ver- 
ticals below  the  points  of  intersection.  Repeat  for 
clusters  at  higher  edge  values. 

Each  vertical  line  either  separates  two  interior  clusters  (thus 
identifying  points  on  either  side  of  the  line  as  to  cluster 
class  name)  or  bisects  an  edge  cluster  (with  points  on  each 
side  being  labeled  by  the  appropriate  class  name  of  the  two 
associated  with  each  cluster) . 

There  are  images  for  which  this  partitioning  scheme  does 
not  assign  a unique  class  name  to  each  partition  region.  Con- 
sider the  drawing  in  Figure  14a  and  its  2-D  histogram  in  Figure 
14b.  When  a partitioning  of  the  histogram  is  attempted,  re- 
gion X is  labelled  by  both  class  20  and  class  30.  This  occurs 
because  there  is  an  actual  anihiguity  in  the  (edge  value,  gray 
level)  classification  space,  since  points  at  gray  level  25 
and  edge  value  20  exist  both  on  the  10-30  boundary  and  on  the 
20-40  boundary.  This  conflict  can  be  resolved  arbitrarily  by 
dividing  X in  half  and  assigning  the  left  half  to  class  30 


and  the  right  half  to  class  20.  Note,  however,  that  disjoint 
partitions  of  the  2-D  histogram  may  be  associated  with  the 
same  class  name.  Perhaps  a better  solution  is  to  avoid 
classifying  points  near  ambiguous  regions  of  the  2-D  histo- 
gram until  it  can  be  done  by  some  other  method  such  as 
proximity  to  already  labelled  points  in  the  original  image. 
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a.  LANDSAT  window  (Monterey,  CA) . 
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2-D  histogram  of  (a)  with  gray  level  as 
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Figure  4a.  Profile  of  a step  edge.  A and  B are 
two  neighborhoods  of  the  same  size 
with  average  gray  levels  a and  b,  re- 
spectively. 

b.  2-D  histogram  (thinned  edge  vs.  gray 
level) . 

c.  2-D  histogram  (thinned  edge  vs.  average 
gray  level) . 
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Fiqure  5a.  Square  on  background. 

b.  2-D  (stretched)  histogram  using 
actual  gray  level  as  x-axis. 

c,  2-D  (stretched)  histogram  using 
average  gray  level  as  x-axis. 
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Figure  6a.  LANDSAT  window  of  Monterey. 

b.  2-D  histogram  (thinned  edge  vs. 
gray  level) . 

c.  2-D  histogram  (thinned  edge  vs. 
average  gray  level)  . 
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Profile  of  an  edge.  A and  B are  two 
neighborhoods  of  the  same  size  with 
average  gray  levels  a and  b,  respec- 
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Figure  8.  2-D  histogram  of  an  image  consisting 

of  two  distinct  homogeneous  regions. 


Figure  9.  Profiles  of  edges.  A and  3 are 
two  neighborhoods  of  the  sane 
size  with  average  gray  levels  a 
and  b,  respectively. 


Figure  10.  2-D  histogram  of  an  image  con- 

sisting of  two  distinct  homo- 
geneous regions. 
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Figure  12a.  Adjacent  object  regions  on  back 
ground  (same  as  Figure  3a) . 
b.  2-D  histogram  for  (a) . 


2-D  histogram  of  Figure  12a, 
partitioned  into  classification 
regions . 


Circular  regions  on  background. 
2-D  histogram,  partitioned  into 
classification  regions. 
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edge  cluster  suggests  its  average  gray  level  as  a threshold.  In- 
terior clusters  may  also  be  defined  as  representatives  of  object 
class  interiors.  The  relation  of  edge  clusters  to  interior  cluster; 
gives  rise  to  a classification  strategy  based  on  partitioning  the  — I] 
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Abstract,  cont'd. 


2-D  histogram  into  disjoint  regions  labelled  as  to  object 
class.  Each  partition  is  a classification  domain  for  points 
of  the  original  gray  level  image. 
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